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THE  PROBLEM 

Most  peer  rating  formats  require  the  rater  to  nominate  a  specified  number 
of  his  group  members  to  the  top  and  to  the  bottom  positions  in  the  group  in  terms 
of  some  particular  trait  or  ability.  When  these  nominations  are  pooled,  there 
are  frequently  members  of  the  group  who  are  not  named  as  either  high  or  low 
(unrated  group).  The  usual  procedure  has  been  to  assign  an  average  (i.e.  mean) 
score  to  such  individuals.  However,  it  seems  possible  that,  for  the  prediction 
of  many  criteria,  the  fact  that  a  man  was  unrated  might  indicate  something  quite 
different  from  '^averageness .  " 

Peer  ratings  of  leadership  potential  have  been  shown  to  correlate  with 
subsequent  success  in  the  Naval  Air  Training  Program.  This  study  investigates 
the  assumption  of  the  "averageness"  of  this  unrated  group  by  comparing  their 
later  success  with  that  of  men  who  received  the  average  score  because  of  having 
received  equal  numbers  of  high  and  low  nominations  (rated  group),  and  with  that 
of  men  who  received  scores  at  levels  other  than  average. 

FINDINGS 

The  results  indicate  that  the  unrated  group  are  not  inferior  to  those  men 
who  were  rated  average,  nor  to  the  average  of  the  total  group  in  performance 
on  the  criterion  used  here. 


INTRODUCTION 


Peer  ratings  or  buddy  ratings,  i.e. ,  evoluations  of  the  members  of  a 
group  by  one  or  more  other  membe  s  of  that  group,  have  proved  to  be  useful 
instruments.  Although  usually  made  by  untrained  and  relatively  unsophisticated 
observers,  such  ratings  have  been  sho\wn  to  be  good  predictors  of  success  or 
failure  in  several  areas  of  endeavor.  Studies  have  indicated  that  such  ratings 
have  substantial  validity  in  predicting  flight  failure  (I),  officer  efficiency  (7), 
military  grades  in  Officer  Candidate  School  (6),  leadership  performance  in 
combat  (5),  on-the“job  performance  (3),  and  scholastic  performance  (4). 

One  of  the  most  common  peer  rating  formats  requires  that  each  member 
of  a  group  nominate  specified  numbers  of  the  top  and  bottom  members  of  the 
group  in  terms  of  a  certain  trait  or  ability.  Since  agreement  betvween  raters 
is  usually  quite  high  (2),  there  often  exists  a  number  of  individuals  who  are 
not  nominated  either  high  or  low  by  the  other  members  of  the  group.  The 
general  procedure  is  to  assign  these  people  a  score  equal  to  the  overage 
(i.e.  mean)  score.  The  question  arises,  however,  as  to  whether  those  people 
not  nominated  are  really  "average"  in  relation  to  the  criterion  of  performance, 
or  are  instead  unique  in  the  light  of  having  not  been  mentioned  at  all. 

Such  peer  ratings  are  among  measures  used  in  the  Naval  Air  Training 
Program  to  appraise  the  potential  of  Individual  cadets.  During  the  eighth 
week  of  training  each  man  in  a  class  of  cadets  is  asked  to  name  the  three 
most  promising  prospective  officers  and  the  three  least  promising  in  his  class. 

It  has  been  shown  (1)  that  these  ratings  typically  have  a  bi -serial  correlation 
of  about  .35  with  subsequent  failure  to  complete  the  training  program  and  that, 
when  combined  with  other  measures,  they  have  considerable  administrative 
usefulness. 

This  study  compares  the  training  success  of  cadets  who  received  an 
average  peer  rating  score  as  a  result  of  not  being  nominated  either  high  or 
low  (unrated  group),  those  who  received  the  average  score  because  of  having 
received  equal  numbers  of  high  and  low  nominations  (rated  group),  and  those 
who  received  scores  at  levels  other  than  average. 

ANALYSIS  AND  RESULTS 

The  peer  ratings  and  completion  records  of  1385  cadets  who  were  in 
the  Naval  Air  Training  Program  during  1958  were  divided  into  six  groups  as 
shown  in  Table  I . 
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Table  I 


Percentages  Completing  Training  at  Various  Peer  Rating  Score  Levels 


Peer  Rating  Score* 

N 

%  Completing 

60  or  more 

163 

81 

51  -  59 

486 

73 

50  -  unrated 

79 

67 

50  -  rated 

57 

60 

40  -  49 

425 

60 

39  or  less 

175 

36 

Total 

1385 

64 

*Standard  scores,  mean  = 

50,  S.D.  =  10 

The  difference  in 

the  percentages  of  rated  and 

unrated  men  with  scores  of 

50  who  completed  training  is  not  statistically  significant* „  However,  the  results 
indicate  there  may  be  a  tendency  for  men  who  had  compensating  low  and  high 
ratings  (rated  group)  to  be  more  like  those  men  who  had  a  slight  preponderance  of 
low  ratings  (scores  40-49)  than  like  those  who  had  a  slight  preponderance  of  high 
ratings  (scores  51  -59).  However,  the  difference  is  not  sufficient  to  be  administra 
tively  useful.  Only  the  lowest  group  (score  of  39  or  less)  has  a  sufficiently  low 
completion  rate  as  to  make  their  early  elimination  from  training  a  profitable 
administrative  action. 

The  data  available  here  do  not  support  the  notion  that  the  unrated  cases 
are  as  a  group  inferior  in  their  training  potential.  Rather  their  completion  rate 
is  at  least  e^ual  to  the  average  of  the  total  group. 


*  Chi-square  test  for  significance. 
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